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This paper looks at curriculum evaluation as it 
operates today and proceeds to examine evaluation in the social 
studies as. viewed by some of the %fell-known authors in tne field. 
This is followed by a review of evaluation used by some of the 
various social studies curriculum projects and presents a number of 
findings. The paper, according to the author, could be seen as a kind 
of informal "meta-evaluation" i.e., the evaluation of evaluation. The 
paper concludes that a formal meta-evaluation of social studies 
curriculum projects is needed if the field is to catch up to other 
disciplines and become more cohesive and worthwhile. Two appendices 
are given: the first contains a listing of curriculum development 
projects in social studies, while the second contains some 
definitions of the term "evaluation". A seven page bibliography is 
also found at the end of the paper. (FDI) 
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If we do not stxive to enrich our traditional 
fragmentary evaluations of selected aspects of 
educadon by continuing reference to some notion 
of wholeness of impact, however vague and 
ambiguous, our evaluations are likely to remain 
at the level of part-time bookkeeping. 
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INTRODUCTION 



"Social Studies, *' as practiced in American schools nas undergone a 
slow uut steady transformation in the last ten years. Traditionally Social 
Studies na? been equated with dull and dry excursions into only two major 
areas of c:)ncern: history and geography. Tue student was requ'red to 
memorize vast mountains of facts and dates, usually retrieved from texc- 
books as dull as tne courses themselves. Although curriculum development 
in the Social Studies continues to lag bebing development in Science, 
Reading, and Viathematics, soma important strides are nevertheless being 
made. These are, for example, now over 40 different curriculum projects 
in opercition in tae United States which are concerned with Social Studies. 
Even more importantly, the term "Social Studies" has been considerably 
broadened out of its old history-geography base to include anthropology, 
economics, political science, and sociology. In addition to the textbook.^ 
which have been vA^ritten to deal with each of the above areas. Social 
Studies curriculum developers have been gradually retreating from the use 
of textbooks as the sole tool for Social Studies instruction. Now slides, 
movies, records, documents, groups of readings, and field studies comprise 
an ever more important part of the curriculum. Despite all the innovations 
in the "New Social Studies" (a term initiated by Edwin Fenton of Carnegie- 
Mellon University), the use of new methods, such as the inquiry method, 
still lags far beh.nd, in large part because teachers are not effectively 
trained to use the materials and methods involved. 

With tne advent of increasing interest in Social Studies curriculum and 
the funding of a large number of Social Studies curriculum projects, the 
problem of evaluation has arisen. In many cases the term seems to have 
left some curriculum developers floundering. The frustration is evident in 
the words of Terry Denny, who, as editor ot an issue of the Educational 
Product ^RePort (FcVuary 1S6S) on educational evaluation, began his intro- 
duction by stating that "evaluation is a worrisome word in educational 
parlance that resists definition about as stoutly as any concept in vogue. 
An inkling of the number and variety of definitions of "evaluation" can be 
zeen from a brief perusal of Appendix I of this paper wi^^ich brings together 
a few of these definicions. Current use:: of educational evaluation include 
improving on-going programs, rating one project against another, assessing 
the merit of a terminated activity, providing counsel for \^ork about to begin, 
seeking valid principles for generalization to other similar efforts and 
recording fully the story of a particular educational effort. From these uses 



Terry Denny. EducationaLP roduct Report, Vol. 2, No. 2 (February 
1969). ^ ^ 
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and others, it vvould be difficult to say tuat there is *'one" definicion of 
evaluation whicn is "best" to use, and it is not fruitful to tr/ and do so. 
Although it is informative to examine the other definitions, it appears 
that definitions of evaluation are "good" in different contexts. 

This paper will look briefly at curriculum evaluation as it operates 
today, will proceed to examine evaluation in the Social Studies as viewed 
by some of the well-known authors in the field, then reviev^ evaluation as 
used by some ox the various Social Studies curriculum projects, and 
present a number of findings. The naper could be seen as a kind of 
informal "meta-evaluation, " a term coined by Michael Scriven to mean 
"second-order evaluation, i. e* , the evaluation of evaluation. " 

Theoretically, meta-evaluation involves the 
methodological assessment of the role of evaluation; 
practically, it is concerned with the evaluation of 
specific evaluative performances. 2 

M^hat is really needed at this point, uowever, is formal meta-evaluation 
of curriculum projects' evaluation programs, not just in Social Studies, but 
in all the other areas of curriculum development where evaluation is being 
used. 

Many of the materials used in this paper came fi'om the Social Studies 
Curriculum Center of Carnegie-Mellon University, and gratitude mubt be 
expressed to tHe two secretaries of the Canter, Mrs. Ethel Strasser and 
Catherine Dudas, for their help and patience. Carol Jones of the Inter- 
national and Development Education Program, University of Pittsburgh, 
typed the manuscript. 



2 

Michael Scrivon. "An Introduction to Meta-Evaiuatioo. " E ducational 
Product Report, Vol. 2, p. 36 (February 1S69). 
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CURRT CULUM EVALU ATION: A GE NERAL SURVEY 
Defining Curricu.'um Evaluatioa 

"Curriculum evaluation" is a tenn w^'ich, by its range of definitions, 
is about as vague as a bank of fog. ': ■•'■■^ component parts of the term are 
equally nebulous. The word "curriculii;-i" in its traditional sense conjures 
up such definitions as " course of study but In fact curriculum now seems 
to have a wide range of meanings, partic ■.•rly v^ith the recent push by sucl, 
people as John Holt, Charles Silberman. ivc-il Postman. Charles V/singartner, 
and Ivan Illlch against the traditional sr, .oot and itc methods. Definitions of 
curriculum have now become very broad. Y'ayne Welch notes that: 

Curriculum is frequently defined by educational tiieorists 
as the link between society and the schools, the major 
source of stimuli found in instructional settings, or as 
pupil behavior pertinent to the goals of the school. 
According to these definitions, curriculum could include 
everything that happens to a c:uld in school from learning 
how to accept last Friday's foouliall game loss to the 
complete sequence oj. experiences tnat produced his 
800 SAT score. 1 

Curriculum is therefore a rapidly changing term and field of study and 
is now going beyond what happens in the school to include experiences 
exterior to the school environment as well. And because the term is broad 
and in a state of flux, il is little wonder that people have difficulty in 
agreeing in what "curriculum" is. Another eysmple of the breadth in the 
definition of curriculum may be seen in che definition presented by Thomas, 
Sands, and Brubaker chat: 

Curriculum means all the intended learning goals, 
experiences, teaching materials and evaluation 
techniques which educators plan and/or use. 2 

Definitions of evaluctic.i also range from the narrowest of focuses to 
extremely broad visions. (A short compilation of some of these definitions 
appears in Appendix I of this paper. ) Melvin Tumin states that: 

Evaluation means many things to many different people, 
not only because they are defensive about the oossible 



^Wayne V/elch. "Curriculum Evaluacion. " Review of Educatio nal 
Research. Vol. 39, No. 4 (1969). pp. 429-4<:3. ~ 

2_ ^. 

R. Murray Thomas, Lester Sands, and Dale Brubaker. Strategies of 
Curriculum Change . Scranton. Pa.: International Texlbook Company, 1968, 



results of a cystematic scrutiny of their effectiveness, 
but bece^Lcs too tney have different notions as to what 
ought to be transpiring in any educational transaction 
and what are the lecjitimate sources of pride and srame. 
So too they have different criteria. . . criteria as to what 
constitutes evidence, . .and they often cannot see how a 
non-participant can possibly^ evaluate as well as a 
participant in an enterprise. 

Curriculum evaluation, then, is and has been just as illustrative as 
its component terms. Ian V/estbury, in a recent and thorough review of 
curriculum evaluation, finds that: 

. . , the assertion that "we must evaluate our curricula" in 
terms of cost, effectiveness, content and tiie like has a 
ring of sense and efficiency and a commonplace obvious- 
ness that makes it impossible to believe the opposite. 
To this extent everyone supports the evaluation of curricula. 
Curriculum evaluation is, however, another thing: it is a 
body of techniques, methodologies, and principles created 
deliberately (and recently) to give some systematic form to 
the vv^ays in which the assertion "we must evaluate". . .can 
be made to work. 

Ronald C. Doll, in the second edition of his well-known v/ork on 
Course Improvement , defines evaluation as: 

• • .a broad and continuous effort to inquire into the effects 
of utilizing educational content and process according to 
clearly defined goals* ^ 

It can already be seen that the definitions of curriculum evaluation 
vary from author to autnor. Such well-known names in the field of edu- 
cational evaluation as Cuba, Stake, Stufflebeam, Cronbach, and Forehand 
all make tneir contributions. The late Hilda Taba perhaps best outlined 
the program of curriculum evaluation when she noted that the term evaluation 
covers an extraordinary vaiiety of meanings and processes. For example, 
one can evaluate anything iibout the schools" curriculum: its objectives, its 
scope, the quality of personnel in charge of it, the capacities of students, 
the relative importance of various subjects, the degree to which objectives 
are implemented, and the effectiveness of the equipment and matenals. 



Melvin Tumin. "Evaluation of the Effectiveness of Education. " Inter- 
change, Vol. 1, No. 3 (1970), p. 106. 
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Ian V/estbury. ^'Curriculum Evaluation. " Review of Educational Rasearcl> . 
4: 240 (April 1970). " ~ ~ 
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Ronald C. Doll. Course Improvement . (Boston: Allyn and Bacon, 1970), 
p. 379. 



S9condly, Taba notei that evaluation may be u^eJ to refer to diiferent 
processes from a rep^aring of a value jud^^ement based on opinion to 
"the careful gatnaring of evidence on the attainment of objectives, a 
forming of judgements on the basis of tnat evidence, and the v>?eigrjin;i 
of that evidence In the light of those objectives. Thirdly, evaluation 
can be carried on at a variety of levels and by different categories of 
people. 

The narrowest concept of evaluation, Taba stated, defined evaluation 
as marking or grading. The wider definition invoWes the process which 
includes the formulation o.C objectives, decisions about the means of 
securing evidenc3, processes of interpretation, and decisions about t':e 
needed changes and improvements in curriculum. ^ Another point of confusion 
in curriculum evaluation is the identification of evaluation with measurement s 
Many authors write a book entitled "Evaluation. . . " and after several intro- 
ductory chapters they wander into discussion of the varieties of measurement 
(intelligence tests, etc. Measurement, of course, is an important part 
of evaluation, but it definitely should not be equated with the larger concern 
of "valuing. " 

Methodological Approaches 

A variety of methodological approaches are used in curriculum evaluation. 
More often than not evaluations are of an informal nature, particularly at 
the local level. Informal evaluation of curricula has a number of drawbacks, 
not notably its lack of oDjective and comprehensive judgement criteria. T.-^e 
generalizability of informal and curricula evaluation is accordingly an 
important and continuing proDlem. 

•.Vhat hd.s been described as a formal evaluation of curricula is 
exemplified by various works of Cronbach, Stevens and Morrisset, and 
others. This type of evaluation often does cet up criteria and objectives 
and can include the decision-making process within its domain. The use 
of formal evaluation is most often found in large curriculum projects and 
offers greater generalizability over a wide spectrum. 

Another approacli to curriculum evaluation t« that of program accounting, 
or cost/bonefit analysis. A prime example is PPBS (Planning, Programming, 
Budgeting System) whicl% while not strictly an evaluative device, is thought 
of as a "system for relating inputs to outputs in such a way as to most 
effectively allocate resources in relation to objectives. " A PPB System is 



Hilda Taba. Curriculum Dev elopment . (New York: Harcourt, Brace, 
and V/orld, 1952), p. 310. 



^bid. 
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a mcinagemeni: an:^ ?lanning tool (rather tiian an accounting tool) for "more 
efficient use of limited resources in the achievement of specified institutional 
goals. The basic concepts of PPBS have been summarized by Freeman: 

1. Explicit identification of objectives based on comparisons 
and anal/ses of alternative objectives. 

2. Systematic examination of alternacive means of 
accomplishing given objectives. 

3. Explicit identification of total costs of each alternative, 
in terms of real costs (money, space, time, manpower) 
and long-term costs of current decisions. 

4. Multi-year planning and programming system. 

5. Organization ol budgets on the bases of objective- 
oriented programs. 

5. Explicit comparisons of costs and benefits of o:^ch program. ^ 

PP3S is therefore not purely an evaluation system £er se; and while it 
has achieved v/ide support from a number of sectors (particularly State and 
Federal governments), it has also been the target for sniping activity from 
those who feel that cost accountability and other such quantifiable aspects 
ignore too many important but non-quantifiable factors. 

Emphases 

Curriculum evaluation has seen a number of different emphases, depending 
on who has devploped the models. Stufflebeam, for excimple, while not 
specifically v\ King witn just evaluatir . ol curricula, emphasizes the 
neglected area of decision-making in many of his studies. His model for 
educational evaluation includes four different components: i. e. , context 
evaluation, input evaluation, process evaluation, and oroduct evaluation 
(CIPP). 9 

Another emphasis in curriculum evaluation manifests itself in the 
evaluation of different aspects of the curriculum entity. A prime example 
of this type is evaluation of curriculum materials , such as textbooks, vjo&- 
b)oks, and teachers* manuals. Another example in this realm is the 



Jack Freeman. PPBS . (Paper presented at the University of Pittsburgh, 
1S70), p. \. 

9 

Daniel Stufflebeam and Egon Guba. Evaluation; The Procr 3ss of 
Stimulating, Aiding, and Abetting Insightful Action. (ColumbusT Evaluation 
Center, Ohio State Uni\ sity, 1968), p. 50. 
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evaluation of various pieces of educational equipment such as tape recorders, 
motion-picture projectors, and teaching macliines. Perhaps the best-known 
example is a rataer expensive journal called Educationcl Product Report , a 
publication of the Educational Products Information Exchange, which provides 
"descriptive and evaluative information about all types of learning materi^tb, 
equipment, and systems. " The information, however, tends to be more 
desciiptive than evaluative. The journal provides many convenient tables 
v;hich allow its readers to compare specifications and costs of various 
pieces of equipment, teytc, and other maierials, but the tables seldom provide 
any comparative judgments about these items. Occasionally separate article£r 
do examine specific curriculum materials in a more thorough manner. The 
descriptive f tevens-Morrisset System has often been employed in these 
analyses. The Report has, from time to time, devoted itself to the problems 
of evaluation. Maurice J. Eash, for example, authored an article entitled, 
"Assessing Curriculum Materials: A Preliminary Instrument. " Eash's 
instrument has the a«-^vantage of being concise and easy to use but like the ,^ 
Stevens-Morrisset system its product would appear to be mostly description. 



3ome Distinction:. 

Several widely recognized distinctions in curriculum evaluation might 
be examined at this point. One is Michael Scriven's "formative-summctive" 
division. Fornaiive evaluation indicates evaluation of a program still in 
progress, while summative evaluation refers* to an evaluation at t^e end of 
a program. Another distinction is that between description and evaluation * 
Arlene Fa/ne maker: this distinction between analyzing curricula to describe 
them, without re ^nce to standards of what they should be, and to evaluate 
them by comparing them v^ith a model or criteria. This distinction is most 
helpful because it appears that a good deal of what is purported to be 
"evaluation" is in reality only "description. "^- 

A third dictinction ' > that between curriculum evaluation and curriculum 
analysis made by Alan Tom. He refers to the examination of the worthiners 
of curricular contents as curriculum analys is, and user the term curriculum 
evaluation when the purpose is to discover Wiiether a curriculunr. ;ciiieves 
wliat it purports to achieve. However a number of writers continue to use 
the term analysis to mean evaluation . Tom feels thac the distinction is an 
important one and use^ a simple analogy to clarify the difference between 
the two processes: 



Maurice J. Eash. "A .-sessing Curriculum Materials: A Preliminary 
Instrument." Educational Pr oduct Report , 2:18-2^ (February 1969). 

^'Arlene Payne. Tiie Study of Cu rriculum Plans. (Washington, D. C*: 
National Education Association, 1969). 
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Suppose wo want to decide whether to use a particular 
insecticide. One question vve would ask is whether an 
insecticide 6oez vmat it is supposed to do, i, e. , destoy 
insects. Yet the questions of v;hether yr/e should use t.-:e 
insecticide is a broader problem. The latter question 
raises such issues as: special equipment needed to 
spread the insecticide, tne effect of t':e insecticide on 
wildlife, the cost of using the insecticide and the effect, 
if any, on humans. 

Tom notes that the distinction has also been recognized by several 
other writers. Iviichael Scriven, for example, has argued that "if the goals 
aren't worth ccl^ieving, then it is uninteresting hov; well they are achieved. . . 
Thus evaluation proper must include, as an equal partner with the m-^asuring 
of performance agains goals, procedures for the evaluation of goalc. 

Still another distinction by Melvin Tumin is that of tne "central polarity 
in the field of evaluation today, i.e. , the dialectic between wholeness and 
fragmentation. " Tumin sees a struggle in the evaluation field 

between trivial precision and apparently rich ambiguity, 
and it is imperative that we strike a better balance between 
these than has been true in the past, l"-- 

Tumin wants evaluators "to strive to enrich our traditional fragmentary 
evaluations of selected aspects of educaUon by continuing reference to 
wholeness of impact. "IS However, he also sees the practical limitations 
of the holists' views (the holists being those who look down i\ eir noses at 
suca things as measures, controls, samples, indicators, and the rest of 
the paraphernalia of scientific evaluation)* He finds that most of the 
important questions in evaluation cannot be answered unless t.'ie evaluator 
is willing to ''accept very fragmented and partial indicators as somehow 
standing for the whole. " The problem, he states, is a result of unavoidable 
disjunction in the human mind between big v^rords and thoughts. "1^ 



Alan Tom. An Approach to Selecting Among Social Science Curricula . 
(Paper presented at Metropolitan St. Louis Social Studies Center, 1970), p. 61 

13 

Michael Scriven. Perspectives of Cunriculum E valuation. {Chicago: 
Rand McNally [AERA Monograph 1], 1967), p. 52. 

14 

Melvin Tumin. "Evaluation of the Effectiveness of Education. " 
Interchange. 1:98 (1970). 
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jProblems In Curriculum Evaluation 

Having looked at tnese four distinctions, or core proDlems, we may now 
look at some of i:e other problems whicii curriculum evaluation faces. A 
major problem, one of definition, comes under the heading of evaluacion 
design. Hav/kridga. for example, defines "design" as "the overall plan of 
the evaluation and tre strategies within eacli phase of the plan. " He 
identifies seven phases in educational evaluation: 

1. Setting up objectives for evaluation. 

2. Selecting objectives to be measured. 

3. Choosing instruments and procedures. 

4. Selecting samples. 

5. Establishing measurement and observation schedules. 

6. Choosing analyi:is techniques. 

7. Drawing conclusions and recommendations. 

These seven components, while not necessarily all-inclusive, bring to 
mind a number of additional problems. There is, for example, the problem 
of objectives,. Gagn6 and Mager feel that objectives must be precise, 
detailed descriptions of student behavior exhibited or the attainment of an 
objective. The v;hole argument on objectives centers on their specificity. 
Many educators feel that behavioral objectives constrict education to 
"trivia' kinds of behavior tnat can be described precisely. " These critics 
feel that the teacher's spontaneity in the classroom may be threatened by 
such objectives. Bloon makes a more moderate view on this problem. He 
suggests that: 

... it is virtually impossible to engage in an educational 
enterprise of any duration without some specification to 
guide one. Insofar as possible, the purpose of education 
and the specification of educational clianges should be .-nade 



17 

David Hawkridge. "Designs in Evaluative Stud;-s. " in Evaluative 
Research. (Pittsburgh: AmRrir.an Institutes for Research. 1970). p. 27 
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explicit if Vaey are co remain open to inquiry, if teaching 
and learning are to be modified or change is needed, and 
if each new group of students is to be subjected to a 
particular set of educational processes. 18 

Havvkridge notes that the setting up of objectives for an evaluation are 
not necessarily the same as setting up objectives for a program, especially 
since the objectives for a program are often set up by somebody other than 
the evaluator. 

Guba and Stuiflebeam have carefully examined one type of design . . . 
experimental design . . .and have described the problems tr.at may arise when 
it is applied to .^valuations. Experimental designs would seem to be 
reasonable methods for evaluation problems "since traditionally both experi- 
mental research and evaluation have been used to test hypotheses about the 
effects of treatments. " Guba and Stufflebeam, however, see four flaws with 
this reasoning: 

1. First, the applicacion of experimental design to evaluation 
problems conflicts with the principle that evaluation should 
facilitate the continual improvement of a program. 

Experimental design prevents rather than promotes 
changes in the treatment because treatments cannot be 
altered in process if the data about differences between 
tioatments are to be unequivocal 

2. A second flaw in experimental design type of evaluation is 
that it is useful for making decisions after a project has run 
full cycle but almost useless as a device for making decisions 
during the planning and implementation of the project. 

3. A third problem with the experimental design type of 
evaluation is that it is suited to the antiseptic conditions 
of the classroom. 

4. A fourth flaw inherent in the application of conventional 
experimental design is the possibility that while internal 
validity may be gained through the control of extraneous 
variables, such an achievement is accomplished at the 
expense of external validity. 1^ 



Benjamin Bloom. "Some Theoretical Iscues Relating to Educational 
Evaluation. " in Educational Evaluation: New Roles, New Means. (Chicago: 
Sixty-Eighth Yearbook, National Society for the Study of Education, 19S9). 
19 

Stufflebeam and Guba, o£. cit. , l<i. 
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The problem oi' c^efiv^.ing criteria for judgin^j the worth of evaluations is 
also of major importance for if inappropriate or insufficient criteria are used, 
the result may be faulty designs and useleso reports. 

In sum, Guba and Stufflebeam have attempted to summarize the problems 
of educational evaluation, including curriculum evaluation, as follows: 

1. Inadequacies of present definitions. 

2. A lack of understanding of the different educational settings 
within which evaluation must oe conducted. 

3. A lack of understanding of generalizable information 
requirements \vi ich educational evaluation studies must 
meet. 

4. The lack of valid structure for tue generalizable parts of 
the evaluation design. 

5. The lack of an appropriate set of criteria for judging tr'e 
worth of evaluation strategies, designs, instruments, 
reports, ecc. 

6. The lack of concepts needed to organize and operate 
evaluation systems. 20 

Several other writers in the field of educational evaluation and 
particularly curriculum evaluation have also suggested the same problems 
plus a number of others. For example, Herbert H. VValberg, looking 
specifically at "Curriculum Evaluation: Problems and Guidelines, " has noted 
the previously mentioned controversy on objectives and the problems posed 
by the educational environment. Walberg carefully examines a problem 
stated in Guba and Stufflebeam* s list: the problem of generalizability : 

That evaluation should be generalizable to specified 
populations of students seems an obvious objective; yet 
most evaluations must be faulted on statistical grounds. ^1 



20 

Ibid. 



21„ , 

Herbert J. V/alberg. "Curriculum Evaluation: Problems and 
Guidelines. " 'reacners CoHeye Record, 71:56': (May 1970). 
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Walberg fault? curriculum evaluation statistically by its r.ack of random 
sampling, thereby limiting inferences to the population Irom which the 
sample has been dra/7n. 



A second part of the generalizabiiity problem involves the changes in a 
course and the students across time. vValbergasks: "To what extent does a 
course remain uncnanged while undergoing evaluation? "22 Formative 
evaluation is, of course, helpful here, but even at the ^tage of summative 
evaluation, the course may still be evolving and yearly cycles of summative 
evaluation may be needed for a few years after the course is completed. 

A third problem involving generalizabiiity across time is 'the changing 
state of society and the possible irrelevance of courses developed before 
relevant changes. "23 



Two other writers have faced the generalizabiiity problem in the 
Educational Products Report : Robert Stake and James L. V'ardrop. Stake 
sees two approaches to educational evaluation. He believes there is a 
choice between being (1) scientific, generalizing and evaluating to find out 
why, or (2) to be descriptive, to be delimited, and to evaluate to find out 
what.^"^* At times Stake thinks that the evaluator should opt for evaluation 
outside the "scientific process, depending on the evaluative job. " He refers 
to evaluation that permits generalization in many directions as "higher 
evaluation, " while "lower evaluation" yields conclusions limited to a 
specific setting. " 

V/ardrop echoes Stake's distinction between evaluation for the "whats" 
and "whys. " He also notes that 

. . . whether or not an evaluation study is designed for 
generalizabiiity (to other classrooms, other children, 
other communities or other times), the consumer {sponsor, 
participant, or interested laymen) will make generalizations 
from its results, even though such generalizations may be 
unwarranted or even illegitimate. The evaluator cannot 
control the ways in which others use the results of his 
evaluation study. 25 



22 

Ibid . 

23 

Ibid . 

24 

tobert Stake. "Generalizabiiity of Program Evaluation. " Educational 
Product Report 2:41 {February 1959). 
25 

James Wardrop. "Generalizabiiity: The Dangers of Limits. " 
Educat ional Product Rep ort 2:42 (February 1969). 
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v,.rArr,„ see. £ - valuators' most eifactlve safeguard on this problem 
as «;fut of t::-n,;;t scientific and most generallzable design possible. 

r r^-sic 1^: T::^^:t=:zfX:z-s ... ... 

can be quite easily applied to curriculum evaluation. 

1. Lack of adequate definition of evaluation, 

2. Lack of adequate evaluation theory. 

3. Lack of knowledge about decisions processes. 

4. Lack of criteria, 

5. Lack of approaches differentiated by levels, 



6. 



Lack of mechanisms for organizing processing and reporting 
evaluative information. 26 



or evaluation model, inis is, oi ^uui^^y oneration, tney 

to^t^ri Lr ethfdec;S-»a.cl„. area. Barbara C- ^^^^^^^^ Isir.'t 
--/different models fl— 

s raTru" it ~ 

ZV'nGuba. "The Failure of Educational Evaluation. " Educational 
Technology 9:29 (May 1969). 

' " 27 ^^,^^^^,33, "Application Of Stulflebeam's Model to ^^^^ 
Program Evaluation. " (Master's Thesis. University of Pittsourgh. 1968). p. .8. 
^^An excellent unpublished (July 1971) descriptive summary of diffe^^^^^^^^^ 

evaluation models which provides information - f/^V°;uon\?bo?atory 
by James Sanders of the Educational 'Research and Evaluation Laboratory. 
Indiana University. Bloomington, Indiana. 

29 Esther Kresh. AnOverviewoltJ^^^ (Pittsuurgh: 
Office of Research. Pittsburgh Public Schools. 1959). 



cost-benefit analysis. In its several years of operation in Pittsburgh, the 
evaluators never got beyond the third stage of the model in actual practice: 
and now that Provus has departed from Pittsburgh, the model is no longer used. 

Conclusions 

Vve have here briefly examined educational evaluation and more 
particularly curriculum evaluation. It has been seen that evaluation can be 
defined in many different ways and that there are both narrow and wide con- 
ceptions of the term. Several methodological approaches can be caken in 
educational evaluation including the PPBS, informal, and formal crpproaches. 
Educational evaluation models often stress different aspects of f\e curriculum 
process. For example, Stufflebeam has emphasized tbe decisior -making 
element of evaluation, A number of distinctions concerning evaluation have 
been made which help to bring the evaluation process into sharper focus. 
These include the distinctions between formative and summative evaluation, 
between evaluation and analysis and between evaluation and description. A 
number of problems plague educational evaluation in addition to the problem 
of definition of terms. Evaluation designs, particularly experimental designs, 
often contain such flaws as lack of external validity and lack of a decision- 
making component. Inappropriate or insufficient criteria may result in faulty 
designs and reporcs. Tae generalizability of an evaluation is also a major 
problem with lack of random sampiino, changes in courses and students 
across time, and the changing state of society presenting particular difficulties. 
Finally the problem of operationalization of evaluation models often stymies 
evaluators at the grass-roots levels. 

Educational evaluation has made a great mdiny advances in the last ten 
years, and its importance is becoming recognijsed at last. Despite these 
advances, it has been seen that a great many problems have yet to be over- 
come. These problems are now being examined in specific academic subject 
areas as well as on a more general scale. The problems enumerated in this 
brief overview of the evaluation of curricula must now be kept in mind as v/e 
move to a review of evaluation efforts in the Social Studies. 
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SOCIAL STUDIE3 EVALUATION: A REVIEvV OF THE LITERATURE 
The Problems 

Vi'ith the "tradinonal" History and Geography-centered Social Studies 
slowly being replaced in American schools, there has arisen a need for 
evaluation of the new Social Studies curricula. Problems impeding the 
development of evaluation include: (1) viev/ing of evaluation merely as 
testing and grading, (2) the limiting of evaluation to information collected 
by teachers, (3) inadequacies of standardized tests in Social Studies, (' ) 
few and mostly descriptive evaluation models aimed specifically at Social 
Studies, and (5) models v/hich disregard die important facet of decision- 
making. These and other problems, as seen in tne current literature on 
Social Studies evaluation, will be examined here. 

Evaluation done by teachers is often considered to be the only type of 
evaluation of importance. Unfortunately, this kind of evaluation usually only 
means "testing" and "grading. " Evaluation by teachers, moreover, often refers 
simply to the measurement of student performance with no particular interest 
in the quality of the curriculum being taught. Some idea of the backwardness 
of this type of Social Studies evaluation can be obtained [torn a survey by 
Dwight Allen and Richard Gross in 1963 w ich reflected the use of the term 
"evaluation" as procedures specifically administered by teachers for testing 
purposes. Six hundred members of the National Council of the Social Studies, 
randomly selected from its mailing list, were asked to reply to 60 questions 
related to three elements of assessment: i.e., what should be covered in 
Social Studies evaluation; the administrative procedures for testing; and the 
kind£, of examinations and test items needed. Some of the conclusions 
derived from the Allen and Gross survey are that: 

1. Teachers frequently fail to relate their assessment 
practices to the aims they claim for their offerings. 

2. Teachers are often inconsistent in their conception 
of evaluation. 

3. Teachers are reticent to use the full range of 
evaluation techniques now available. 

4. The use of many evaluation devices is misunderstood 
and such devices are often misused. 

5. All of the purposes of evaluation are not understood 
by many teachers. 
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6. Teachers almost unanimously accept botn essay and 
objective te^t items. 

7. A disproportionate amount of time seems to be spent in 
the correction of English errors in social studies work. 

8. The theory of sampling and test construction is not 
understood by teachers. 

9. Few teachers employ item analyses or other checks 
upon their teaching, and their evaluation procedures. 

10. Teachers, by their practices, encourage students to 
regard grading as a coercive weapon to be used against 
them. 

11. Very few teachers perceive the major implications of the 
evaluation program which carry beyond the grading of 
students. ^ 

This survey is interesting because it shows that Social Studies evaluation 
by teachers is indeed backward. But secondly, despite the fact that the 
Allen and Gross survey concerns 'Troblems and Practices in Social Studies 
Evaluation, " the conclusions of the survey mostly seem to refer only to the 
role of the teacher in evaluation and this role was viewed essentially as 
testing and grading. The authors of the article from which the survey 
results were tsken note that '^evaluation should be used in a more compre- 
hensive sense. Lr establishing realistic aims, in setting up purposeful 
curricula and courses, in critically reviewing instructional materials, and in 
judging the efficacy of our teaching process. "2 But their survey questions 
imply an unfortunately narrow view of evaluation. A broader survey of Social 
Studies evaluation is needed, one that will include evaluation procedures used 
in some of the major Social Studies curriculum projects, as well as "evaluation 
of evaluation, " or meta-evaluation. 

Gross and Allen do, nevertheless, note a few evaluation problems wliich 
they believe are special to Social Studies evaluation. Among them are: 



Dwignt Allen and Richard Gross. "Problems and Practices in Social 
Studies Evaluation. " in T eaching the Social Studies, by Richard Gross. 
W. McPhie, and Jack Fraenkel. (Scranton, Pa.': International Textbook 
Company. 1969), p. 480. 

^Ibid. 
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1. Tne broad and imorecise goals that are held for tne 
Social Scuc'ies. . . plus the future-oriented socio- 
civic purpose: commonly expected to result from 
social education. 

2. The problem ol validity. Do tae instruments we attempt 
to apply in evaluation really ascertain what we believe 
tiiey do? Even as wa build tiie necessary measures of 
skill we remain dogged by a douoc t .at nas not been 
alleviated to dace: are such tests of competency 
actually measures of the quality implied or are they 
largely indexes of abilit/ to apply the knowledge 
already learned? 

3- Among tne major purposes of Social Studies goals 
are those reflecting desired socio-^ivic attitudes. 
Can a purely objective social science approach 
ensure progress in this domain? 

'•i. The heart of the Social Studies program should rest 
in controversy. Often in controversy there is no 
answer or t'-ere is the possibility of a variety of 
hypotiieses that either can't all be tested or that 
v/ill not satisfy many who are involved- 

5. Unhappily, the great buJJ; of commercial and 
standardized tests in the field of social studies 
have been found wanting b/ experts. ^ 

Although Gross and Allen see the above problems as specific to Social 
Studies, the validity problem certainly is not nor is tne problem rf devising 
tests which measure quality instead of previously learned knowledge. And 
difficulties with standardized tests -Iso plague other fields as well. This 
brings to light a major problem among writers on Social Studies evaluation: 
they do not seem to take note of what ii. happening in other fields as far 
as evaluation is concerned. 

Something of the slow progress in Social Studies evaluation is evidenced 
in the 19G5 Yearbook of the National Council of the Social Studies, 
••Evaluation in Social Studies. " A/Vhere this anthology leans heavily toward 
"measurement" and "testing, " a more recent yearbook, edited o/ Dorotiiy 
McClux-e Frazer, contain;; a more mature look at evaluation. 



Ibid., pp. -.80-181. 
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Handbook of Research on Teacuing has noted, research on 
evaluation, done especially in regard to Social Studies, over tr.e last decado 
has been limited. "The need is acknowledged but che responsibilit/ is left 
to professionals in evaluation. Unfortunately, there has been little 
activity by professional evaJuators in the area of Social Studies. 

Social Studies Evaluation Models 

Perhaps one of tne most practical efforts at the development of an 
evaluation model for Social Studies education is that of William Stevens. Jr. . 
and Irving Morrissett. 5 Called "A System for Analyzing Social Science 
Curricula. " it oflers a method for systematically analyzing curriculum 
materials and includes an extensive checklist of questions >Arhich usert; 
might ask of materials. Their outline has Kix headings: (1) Descriptive 
Characteristics. (2) -Rationale and Objectives. (3) Antecedents Conditions. 
(4) Content, (5) Instructional Theory and Teaching Strategies, and (6) Overall 
Judgments. The Stevens and Morrisset model, being one of the few usable 
evaluation systems for tl-e Social Studies, has been used more and more of 
late. Unfortunately, there is no concrete evidence concerning how well the 
model works. The authors have noted a few of the comments on their own 
model in an article in the EPIE Forum . 5 One of the criticisms that Stevens 
and Morrisset themselves take seriously is that their model is too unwieldy. 

M. Trances Klein and Louise Tyler, in a critique of the Stevens and 
Morrisset model, however, find the complexity of the model as an asset. 
They see the system's value largely in the detailed comprehensiveness of 
the categories and questions formulated. They found that it "was the most 
comprehensive sec of guidelines dealing with the analysis of curricula that 
the vmters have seen. " Klein and Tyler, moreover, note that several 
questions in the Stevens and Morrissett system suggest important standards 
for evaluating a social science curriculum: 

^' Behavioral Objectives; Does the author word his 
specific objections in such a fashion that the verbs 
demonstrate student action-bohavior that is clearly 
observable and/or measureable? Are specific guides 
to observation and measurement given? 



N. E. Gage. Handbook of Research on Teachi ng. (Cnicago: Rand 
McNally, 1963), p. 

5 

Dorothy McClure Frazer (ed.). Social Studies Curriculum Development . 
(V^ashington, D. C. : National Council ol the Social Studies, 1S3S). 

g 

V'.illiam Stevens and Irving MDrrisset!:. "A System for Analyzing Social 
Science Curricula. " Tl'.e EPIE Forum. 5 (Decemoer 1967), p. 15. 



2. Articulation : Do the materials fit well with th^ 

existing curricula that wil precede and follo\v them? 
Do thsv 'it well with materials in other subject 
studieb wimuJtoneously?^ 

The fiict questions on behavioral objectives, according to Tyler and 
Klein, infer a standard regarding the de^ireability of beh^^aorly defined 
objectives while the second set of questions on articulation suggests "a 
standard which would ensure the new materials were compatible and con- 
sistent with existing and future curricula. "8 

Hov^ever, Tyler and Klein also point out a major fb^ in the Stevens and 
Morris r.ett evaluation system. It is a flaw related to One of tiie distinctions 
discussed in the first part of this paper, namely tiiat bc?tween description and 
evaluation, and it definitely limits the model. SteveQs ^nJ Morrissett never 
state that their system is useful only for descriptive Purposes. Tyler and 
Klein examine this flaw: 

Application cf the set of taxonomic questions wouy result 
in a very extensive, detailed description of a curri/;uium. - - 
assuming that one could obtain ansv/ers to all or ttiO^t of 
the questions. The results would not directly anSv^er 
specific questions reoarding what curricula should b^. 
Many judgments might be inferred from the answer.^ to 
ihe questions, buf there is no specified criteria a^^i^st 
which to formulate ;udgments regarding the quality of 
the curriculum. S 

Scriven takes Stevens and Morrissett to task for Some of the same 
reasons as Klein and Tylei. . . plus a few others. First of all, Scriven finds 
that the Curriculum Analysis System, in both its long ^nd short forms, has 
••an absolutely perva^-ive sense of confusion about evaluation and description. 

Take, for example, t!us description under 2. 2 (General 
Objectives): V.'hat are the generalized student outc^m^s 
that can be expected from the use of tl-ese materials? 
Obviously this calls for an evaluation and a crucial One. 



7^ 

Frances Klein and Louise Tyler. "On Analyzing CuH icuiar Curriculum 
Jhepry_I\iety/qrk, (ripiiuo p. 23. * 

^ Ibid . 
^Ibid. 
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It must be mada clear whether this Is what the producer 
says, or whether it Is, m fact, part of the evaluation 
tucked into the f^ationale section. 10 

Scriven also faults the Curriculum Analysis System for its commitment 
to the Bloom and Krathwohl taxonomy of objectives, wHich he no:es has 
been shown to be seriously defective. He concludes that the Stevens and 
Morrissett system is "too committed to methodological and physical doctrines 
of excessive dubiety, excessively complicated, repetitious, and imprecisely 
described. "^^ As this is the major evaluation system used in Social Studies 
education today, it would seem that Sociel Studies curriculum developers and 
evaluators are leaning on a "weak reed, " and many do seem to use this 
system. 

Aside from ihe Stevens-Morrissett system, the choice of evaluation 
systems specifically for Social Studies is very slim indeed. Dennis Cooler, 
in the Educational Product Report (October 1969), has taken Robert Stake's 
general evaluation model and tried to tie it into the evaluation of Social 
Studies curricula. 12 Ho'wever, this model, too, as it is used by Cooler, 
is more descriptive than evaluative in nature. As Barbara Cass has indicated 
In her examination of seven evaluation systems, the Stufflebeam model (CIPP 
Model) should be mud: better in this regard. 13 stake's model is also a good 
comprehensive descriptive model and should not be ignored. His distinctions 
among antecedent, transactional, and outcome data are very helpful. He 
defines antecedents as any conditions existing prior to teaching and learning 
which may relate to outcomes. Transactions are the encounters of students 
with teachers, student with student, author with reader, parent with counselor, 
. . . the succession of engagements which comprise the process of education. 
Stake describes the outcome as a body of information including measurements 
of the impact of instruction on teachers, administrators, counselors and 
others, data on use of equipment, effects of the learning environment and 
cost incurred. 1'* 



^^Michael Scriven. "An Introduction to Meta-Evaluation. " Educational 
Product Report . 2 (February 1969), p. 38. ~~~ ' 

^^Ibid. 

12^ ; ^ 

Dennis Cooler. "Evaluation and Change in the Social Studies. " 
Educationa l Product Report. 3 (October 1369), pp. 6-13. 
13„ ^ 

Barbara Cass. Application of Stufflebeam' s Model to Large-Scale 
Program Evaluation. (Master's Thesis, University of Pittsburgh, 1968). 
pp. 37-38. 
14 

Robert Stake. "The Countenaace of Educational Evaluation. Teachers 
College Record. 68 (April 1967), pp. 523-540. 
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Another and racier extensive look social stu?ief evaluation has been 
done by Poland F. Payetce and C. Benjamin Cox in an article entitled, "N^w 
Dimensions in Evaluation of Social Studies Programs. ^'15 xhev also use the 
Stake model to a^ses. evaluation in the social studies, All.iough the Cox and 
Payette work is detailed and quite conipre: ensive, it shares the Stake model's 
major flaw; i. e. , a total neglect of curricular decision-making. Payette and 
Cox admit that their dosign ^viil not take care of this critical area with the 
qualification t!iat "it will not detail a step-by-step account of how to evaluate 
a social studies program as a means to making particular curricular decisions. "1 



Evaluation of Social Studies Materials 

Other evaluation models in the Social Studies seem to aim particularly 
toward the evaluation of materials such as textbooks and equipment. These 
evaluations range from the quite systematic to the highly subjective. A 
prime example of bias in this type of ''evaluation" is found in the Textbook 
Evaluation Reports issued by the Textbook Evaluation Committee of America's 
Future, Inc., from Nev; Rochelle, New York. According to the "Committee, " 
it came into being "because of much evidence of socialist and other propaganda 
in textbooks currently used in our secondary sciiools. " They state that its 
purpose is: 



... to evaluate these i-extbooks to determine the accuracy 
and competence and to report objecUvely the extent to which 
they give misleading or false impressions about our American 
form of government, our unique economy, our history and the 
relationship of the United States and its citizens to other 
countries and peoples of the world. The reviewers will also 
recommend textbooks which meet the Committee's standards. ^'^ 

Unfortunately, it is a little difficult to tell what the Committee's standards 
are, since they never seem to be stated anywhere, but the political bent of 
the evaluations becomes obvious immediately. In the review of a social 
studies book (1955 vintage) called Living in Our Democracy, by Vanza Neulsen 
Devereaux and Homer Aker, the reviewer, Medford Evans (a former chief of 
training for the Atomic Energy Commission) begins by stating that: 



Roland Payette and Benjamin Cox. "New Dimensions in the Social 
Studies Programs. " in Social Studies Cur riculum Development: Prospects and 
Problems, ed. by Dorothy McClure Frazer. C/'/ashington, D. C : National 
Council of the Social Studies, 1S69), p. 20S. 



Textbook Evaluation rieports . (New Rochelle, N. Y. : Textbook 
Evaluation Committee of America's Future, Inc., 1955), p. 1. 
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* . . this text is not bad as they go but it just sl.ows how 
bad they go. It does not plug the Communist Party Line, 
as textbooks too often do, and indeed includes truly 
patriotic expressions; yet basically socialistic assumptions 
are taken Tor granted, so that following this test the 
conscientous teacher and pupil will plod ever deeper 
into socialist territory, where annexation by Communism 
ma/ occur at almost any time. 18 

It is this type of biased review which has prompted Malorie Edelson to 
urge the production of "more objective and dependable textbook analyses. "19 

One attempt at iihe production of an objecUve evaluation instrument is 
the "Preliminary Instrument for Assessing Curriculum Materials, by Maurice J. 
Eash, whic:i is quite detailed and comprehensive. The model is centered 
around four constructs: 

1. Statement of Objectives 

2. Organization of the material 

3. Methodology (the modes of transaction used for 
engaging, focusing, and directing the learner) 

4. Evaluation (guiding the learning through feedback 
as well as yielding data on the accomplishment of 
objectives. 20 

Descriptive in nature, thi;. instrument snould be most helpful to school 
boards and others v/;io must choose textbooks for school systems. The 
Education al Product Report is also a valuable source in this regard. 

A number of valuable guides to Social Studies curriculum projects are 
published by tiie Far V/est Laboratory for Educational Research and Development 



Ibid. , p. 2. 

19 

Maiorie Edslson* "Evaluation and Confrontation: Keys to Textbook 
Change. " Educational Product Report. 3 (November 1969), p. 6. 

20 

Maurice Lar.h. "Assessing Curriculum Material^: A Preliminary 
Instrument." Educational Product Report. 2 (February 1959), p. 18. 
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in Berkeley, Califoraia. The reports concicaly examine the various projeccs 
in terms of Goals and Objectives, Content and Materials, Clascroom 
Strategy, Student and Teacher Prerequisites, Implementation, Requirements, 
Costs, Project History and Evaluation. Again these reports are descriptive 
in nature and quite helpful. 

Conclusions 

It can be seen that, outside tae v/ork of Taba and Stevens and 
Morrissett, little attention has been paid to the production of evaluation 
systems geared specifically to oocial studies. ?obert Stake's model seems 
to have been adapted by several of the authors cn social studies evaluation 
but the Stake model tenJs to ignore the important facet of decision-making. 
Most of the other materials related to evaluation in social studies are either 
purely "descriptive" or aimed at the assessment of social studies materials 
only. In other words, it must be concluded that evaluation, at least as seen 
from the literature specifically dealing wii:i social studies evaluation, is a 
sad state of affairs indeed. 

Is it necessary, hov^ever, to have an evaluation system aimed just at the 
social studies? Probably not. If social studies evaluators could use or 
modify other comprehensive models such as Stufflebeam's CIPP model, social 
studies evaluation may show marked improvement. 

Having looked at the literature of Social Studies evaluation as it appears 
in recent books and journals, it must be noted that there is another very 
important source of evaluation literature, and this is from the Social Studies 
curriculum projects presently in operation in the United States. At this 
point, I would like f-j hripfly oxamine evaluation as it is carried out in some 
of t-hfis^ projects* 
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EVALUATION AND THE SOCTAT. .'STUDIES PROTECTS 

There are betv/een 30 and 40 Social Studies curriculum projects now in 
various stages of operation in the United States (see Appendix A). As Edwin 
Fenton has noted, the trickle of materials from these projects should soon 
reach flood proportions. Projects cover all aspects of the new Social Studies 
curriculum including history, geography, anthropology, economics, and 
sociology. Each of the projects, however, takes a different view of what 
evaluation should be. In this section of the paper, I will review evaluation 
problems and procedures carried out in nine different Social Studies projects 
currently undenvay. 

Evaluati on Problems in Curriculum Projects 

Educators increasingly look to curriculum projects as the accepted medium 
for preparing classroom materials and for speeding up the innovative process 
in the curriculum. Projects are sponsored through universities and colleges, 
professional organizations such as the American Geographical Association, 
and non-profit educational organizations such as the Educational Develooment 
Corporation, Inc.. in Cambridge, Massachusetts. Hu Ida Grobman has noted 
some of the problems which face curriculum development projects generally 
in an excellent AERA monograph.l First, as mentioned earlier, curriculum no 
longer is synonymous with the textbook and as curriculum projects have 
broadened their views of what curriculum includes, the scope of many of the 
evaluative investigations has also expanded. 

Grobman also notes an often neglected aspect of evaluation in curriculum 
projects, that is the politics of evaluation, . . the extent to which systematic 
evaluation is politicelly feasible, that is the extent to which the project can 
afford the consequences of evaluation. " In addition., there is the problem of 
the ethics of evaluation, a concern often ignored by writers on social studies 
evaluation. Information for evaluation often can be obtained either without 
the subject's knowledge that the information is being obtained or without his 
knowledge of the use to which it will be put. For example, some researchers 
have used "anonymous questionnaires" that are not really anonymous or 
interviews that are ostensibly to be used for one purpose when in actuality 
they will be used for another. The dubious justification for all this is that 
it is "effective. " 



Hulda Grobman. Evaluation Activities of Curriculum Projects . 
(Chicago: Rand McNally, AERA Monograph No. 2. 1970), p. 3. 
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Grobman has observed that the kind of evaluation and the nature of the 
evaluation design are influenced by what is possible at a particular point in 
the project, when the project information will be used, the purpose for which 
it will be used, who will use it and the purpose of the whole project. \\^it\-) 
these points in mind, we may now turn to an examination of some of the 
social studies curriculum projects. 

Evaluation in the Social Studies Projects: A Review 

The Holt Social Studies Curriculum Project {.as been developed by Edwin 
Fenton of the Carnegie-Mellon University Social Studies Curriculum Cenier 
in Pittsburgh, Pennsylvania. The project began in 1963 and the materials 
referred to here were first used in 1964. However, the Social Studies Curriculum 
Center continues to produce a great number of new materials each year. The 
overall goal of this curriculum project, in the words of Fenton, was "to help 
each student develop to the limit of his ability into an independent thinker 
and a responsible citizen of a democracy. " The program's specific objectives 
in terms of learning and development by students were: (a) the attainment of 
affective goals, the formation of attitudes which will encourage intellectual 
curiosity and independent inquiry, and the examination of values; (b) the 
acquisition of inquiry skills; and (c) the learning of content that is based 
on the latest scholarly knov^ledge. 

The Holt (Carnegie-Mellon) Social Studies Curriculum has yet to be 
systematically evaluated. According to the Far West Laboratory for Educational 
Research and Development, which has reviewed the project, 2 the curriculum 
underwent classroom trial in five Pittsburgh schools and war evaluated on the 
basis of standardized tests and a Carnegie-designed Social Studies Inquiry 
Skills Test. In addition, a sufc)jective questionnaire was used by the developers 
to record student evaluations. The Carnegie-Mellon staff briefly evaluated 
the first experimental courses for the ninth grade using STEP and Economic 
Understanding tests, but this evaluation, of a formative type, produced little 
usable data. It is somewhat interesting to note that under pressure from 
teachers who felt that the standardized tests were not measuring certain 
important skills, the Carnegie group designed their own Carnegie Test of 
gogjgi^Sludies Jn^ Skills, which showed that experimental classes did 
better with the Carnegie materials. One might, however, be justifiably 
suspicious of an "in-house" developed test used to value in-house materials. 



Note: Descriptions of the nine projects in this chapter were taken 
from reports issued by the Far West I^bor^^tory of Educational Research and 
Development, Berkeley, California. 
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Another study of the Holt-Carnegie materials, specifically the curriculum 
on Comparative Economic Gystems. was evaluated by the NDEA In stitute in 
History for Curriculum Specialists using the Stevens-Morricsett Curriculum 
Analysis scheme. Far est Laboratory notes that the Comparative Economics 
and the Comparative Political Science course are similar and that the con- 
clusions of the Stevens-Morrissett model in this case may be the same: 

1. The materials will probably be most useful to average 
and above average ability students. College-bound 
students would be most likely to achieve most of tlie 
objectives. 

2. Below average or culturally deprived students, under 
the direction of a creative teacher, could achieve 
more objectives at a higher level of achievement 
through the audio-visual aspects of the course and 
through a conventional economics (or civics) course. 

3. Minimum skills required of students are: ability to 
read at or near the ninth grade level, computational 
skills (for economics), and interpretive skills. 

At this point the reader must be reminded of the heavy criticism of the 
Stevens-Morrissett model. As mentioned earlier, Scriven and Taylor and 
Klein have all showed that the scheme would produce descriptive results 
only, which it seems, judging from the vague conclusions above, is what 
happened when the model was applied to the Holt Curriculum. There does 
not seem to be any information on how this program compares with other 
programs or with any set of standards in terms of student effectivene£s. 

A second Social Studies project is that of the Educational Research 
Counc il (formerly called the Greater Cleveland Research Council) under the 
direction of Raymond English. The materials, once known as the Greater 
Cleveland Social Science Curriculum, will be published in final form by 
Allyn and Bacon. The Greater Cleveland Social Science Program began in 
1961. The staff of this project voiced certain objections to traditional 
Social Studies programs such as the fact that traditional teaching relies too 
much on rote learning, that traditional curricula view Social Studies as bodies 
of content, that the teaching of Social Studies is too teacher-centered, and 
that texts are too fact-centered. Even after a decade's work it is still 
difficult to tell how well this curriculum, which attempts to remedy these 
complaints, will accomplish its objectives. By 1969 the new curriculum had 
been completed for the ninth grade level, and completion of the tenth, 
11th, and 12th grade materials is scheduled for 1972. But, no formal evaluation 
procedures or tests of the program have been developed. Insi.ead, the 



developers have established a set of informal evaluative procedures, used 
in Cleveland area schools^ which center around questions such as: 



vVi'iat concepts are too easy? Too difficulc? 

Is the program geared Tor the above average pupil? 
For all pupils ? 

What methods are most effective in developing the 
concepts ? 

Is the amount of material sufficient? 
Is the material relevant? 

So, despite a change of name, the project's evaluation system remains 
the same: vague, v/ith subjective, informal attempts at formative evaluation 
and no firm plans for summative evaluation as yet. No comparisions with 
other curriculum projects have been made* 

The Utah State JJnJversjtx Soci al Studies Project is under the direction of 
James P. Shaver, who, like Fenton, is one of the leading lights of social 
studies curriculum development. Located at Utah State University at Logan, 
Utah, the project is funded by the U. S. Office of Education and materials 
resulting from the project will be published by Houghton-Mifflin Company. 
The initx^l proposal for the project was submitted by Dr. Shaver in 195S. 

The developers set the follov/ing objectives for the project: 

1. To develop an Outline of Concepts appropriate to and 
usable in the analysis of public issues. 

2. To develop suggestions lor teaching the concepts in the 
Outline. 

3. To investigate interactions betv;een different disc ssion 
styles and student personality traits as they affect 
learning of the concepts in the Outline. 

The curriculum was designed for average to above-average students 
in grades 11 and 12 but the developers now state that the materials can be 
used at any secondary level. 

The Utah materials were evaluated twice in two different Utah communities. 
The first field test involved 212 eleventh-grade students divided into four 
sections with two being taught by project staff and two by local non-staff 
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teachers- Tuere was no control group- However, at the time of this first 
evaluation, the primary concern of Shaver's group was to study the relation- 
ship between teaching style and student personality, not evaluating the 
curriculum and its materials. Apparently, the project staff felt that, in 
their viev; at least, the evaluation of the earlier Harvard Social Studies 
Project (1964-67), which served as a model for the Utah Project, proved 
that their approach worked; (Shaver and Donald W, Oliver collaborated on 
the Harvard Project) and so, therefore, it was believed that further evaluation 
was not needed. The results of the second field test of the Utah materials 
have not as yet been reported. Even rhe reports of tne assessment of the 
first field test were termed inconclusive. All in all, then, there is actually 
little evaluative evidence available about this project. 

The Harvard Project Public Issues Series, which is now terminated, was 
directed by Donald W. Oliver, a Professor of Education at Harvard University, 
and was funded by the U, S, Office of Education. The Public Issues Series 
is published by American Education Publications and became available in the 
fall of 1970, The Harvard materials are designed for secondary-level students 
of average to above-average ability, and the teacher using the materials has 
to have substantial knowledge of controversial issues. In regard to 
specific objectives, tne Harvard materials are designed "to teach students 
analytical skills that will enable them to discuss public issues more 
effectively. " The content emphasis is on public issues and not on specific 
disciplines. 

According to the Far V/est Laboratory for Educational Research and 
Development, the Harvard Project has carried out informal clinical evaluation 
of ongoing teaching and systematic evaluation of the program at its termination. 
No information seemed to be available on the informal evaluation,and the 
formal evaluation consisted of rating and content evaluation of oral dis- 
cussion and "fairly conventional paper and pencil tests, " Field tests of 
the Harvard Project materials were carried out in Newton, Massachusetts, 
High School from 1964 to 1957 with 46 students. Three control groups were 
used. The project evaluator has been quoted as stating that the evaluation 
design had two major weaknesses: 

1- Students were not tested before or during the three- 
year program, 

2s All testing was concentrated into a tv/o-week period 
at the end of the school year. 

The evaluator concluded that "the one-shot nature of the testing justifies 
some caution concerning the validity of the results, " That would be an 
understatement, judging from the reports on evaluation in this project. 
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The Project staff constructed three written tests to measure learning 
outcomes relevant to the Project curriculum: a Concept Application Test, a 
structured Dialogue Analysis Test, and an Open-ended Dialogue Analysis 
Test. Also the Project da'. elopers constructed an open-ended test to compare 
students' retention of factual information about major American History 
topics. Additionally, the students took the standardized Educational Testing 
Service Problems of Democracy Test. The second Dart of the formal 
evaluation involved "dividing the students into groups of two and then 
asking them to discuss a fairly complicated case study they had read. 
Students were compared on the basis of these discussions. Although the 
results of these cests were quite favorable to the Project, the weaknesses 
mentioned above tend to destroy any sense of confidence one has in this 
partial evaluation. 

The Law_i,n Ameri can Society Project wrr developed by the Law in 
American Sccietv Foundation under the direction of Robert H. Ratcliffe. The 
Project was jointly sponsored by the Chicago Board of Education and the 
Cnicago Bar Association, and the mat-rials became available from the 
Houghton-Mifflin Company in the fall of 1370. The curriculum is based on 
a series of materials entitled. Justice in Urban America , in which the 
emphasis is on how the law operates in the United States. . . what it really 
does in everyday life and what it can be made to do. The Project wants 
students to look at their problems In a lawyer-like way and to help them 
to deal with the problems of their communities as well as broad contemporary 
issues. The developers divided their specific objectives into four groups- 
1 cognitive skills, (2) attitudes and values, (3) the process of inquiry, and 
(4) facts, principles, and generalizations from the discipline of law. The 
curriculum materials were designed for ninth graders but are believed to be 
suitable for all secondary students. 

Little information seems to be available about the results of the 
eva uation which consisted of pre- and post-tests administered to ninth- 
grade classes who used the materials, in Chicago's Title I-supported schools 
and to a control group. The tests were designed to measure "students' 
growth in basic compreiiension of the subject matter and attitude toward the 
function of the legal system, individual influence on the legal process, 
purposes of law enforcement, etc. " In addition to these tests, other 
evaluation measures included classroom observation and questionnaires for 
teachers. The materials presently are being taught for the first time in a 
f'.U-yoar course, but no information on results is available. 

m.fPr-''.!rT^l~°-^^'^ emphasis is on American History,and the 

un 'er S developed by the Committee on the Study of History, 

arade Wpi'''?h ° curriculum is for any secondary 

^ublf^M r "^"''''"'^ ^""""^^ the Addison-Wesley 

Publishing Company in the fall of 1970. 
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Field testing of each unit of tlis Project is determined by the number of 
f^acher requests and by the number of copies available. Most of the units 
were tested in urban areas. The Project utilizes pre- and post-tests con- 
structed by Drs. Rose Olver and David Schneider, who are psychologists at 
Amherst College. The tests attempt to measure "student control of major 
ideas and concepts" and include attitude scales, short-answer questions, 
semantic chacklists, and reading-skill and interpretation questions. The 
Far West Laboratory notes that the tests are still in the developmental stage 
and that no hypotheses or criteria for field testing past or present have been 
published. In addition to this evaluation, teachers using the materials are 
required to complete daily logs and evaluate each unit to determine if it is 
clearly explained, if enough background material is provided and what might 
be added to make the teacher's manual more effective. Also, Amherst staff 
representatives visit every teacher using the materials at least once a year 
and report on classroom atmosphere, teacher's role and style, context of 
the class and lesson, and so forth. It is unfortunate that no published 
conclusions are available regarding this project's evaluation either. 

The Lincoln-Fi lene Center Secondary Social Studies Program is designed 
for students of below-average to average ability levels. Its major content 
emphasis is on political science. Its developer and publisher is the Lincoln- 
Filene Center for Citizenship and Puolic Affairs, located at Tufts University, 
Medford, Massachusetts. The developers have two major objectives. The * 
first is the affective objective of "helping each student increase his personal 
sense of political, social, and economic efficiency. " The second objective 
is of a cognitive nature; that all citizens must learn the ways they can affect 
their environments. The curriculum materials, aimed at non-college bound 
students, also became available in the fall of 1970. 

The evaluation of the narratives in the program is based on written tests 
and feedback from teachers and students. "The student materials are judged 
against three criteria: relevance to stated objectives; interest; and 
readability. " The evaluation of the first criterion is based on objective 
tests (cognitive objectives), subjective tests (affective objectives), and 
informal feedback from students. "Interest" is determined by written 
evaluation and informal feedback from students, and "readability" is 
determined on the basis of content and vocabulary tests, evaluation by 
reading experts, and subjective observation by teachers. 

Although there has been no "hard" evaluation of the program, subjective 
evaluation seems to show positive affective changes as a result of applying 
the curriculum. 

Sociologic al Resources for the Socia l Studies is a project developed by 
the American Sociological Association and is under the direction of Robert C. 
Angell. The curriculum materials emphasize sociology and are aimed at 
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secondary students of average to above-average ability. The publisher of 
the materials is Allyn and Bacon, Inc. The project emphasizes involvement 
of the student in the process of inquiry, and its objectives, which are stated 
in general and not behavioral terms, concern such topics as the Role of 
Modern Women, Leadership in American Society and Analyzing Modern 
Organizations. It is anticipated that the materials will be completed about 
September 1971. 

Evaluation of each unit or episode is done by means of various tests, 
interviews, questionnaires, and observation of trial classes. The tests 
used are a verbal ability test from the Psychological Corporation and a test 
designed by the project staff to measure students' mastery of the content of 
each episode. Teachers using the materials are given interviews and asked 
to complete questionnaires about an episode's effect. Classroom observation 
was also done. Field trials of tiie materials have occurred in Atlanta, Miami, 
Minneapolis, Seattle, and San Francisco. The Project does not yet have 
evaluation results for all of its episodes nor is there a comparison of the 
results of different episodes. Initial reaction to individual episodes is 
claimed to be favorable, however. 

The last curriculum project to be reviewed is the High School Curriculum 
Center i n Government Project developed by the High School Curriculum Center 
in Government at Indiana University in Bloomington, Indiana. The director 
of the project is Howard Mehlinger. The content emphasis is political 
science, and the materials, not as yet formally published, are designed for 
average to above-average secondary students. The developers' objectives 
concern the students' political knowledge, skills, and attitudes. 

This project has developed one of the most comprehensive evaluation 
schemes of any of the above projects. John J. Patrick, co-author of the 
materials for this project with Howard Mehlinger, has written an excellent 
report on the evaluative procedures used. The project's materials were 
developed during a three-year period from 1966 to 1969, Two field trials 
of the materials have been conducted to date: during 1968-69 and 1969-70. 
Further evaluation study will be conducted during 1972-73, when the materials 
will be first used in published form. The first field study relied only upon 
feedback from questionnaires, classroom observation and students' test- 
answer sheets. The second evaluation study used more formal procedures, 
including a Political Knowledge Test, a Political Science Skills Test, and 
six political attitude scales, to measure student performance in terms of 
the instructional objectives of the course. The conclusions of this 3tudy 
appear quite favorable, and extensive rewriting of certain parts of the 
materials are taking place to correct defects discovered during the second 
field trial. 
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Conclusions 

This cursory review of nine Social Studies projects now currently in 
operation across the United States covers materials published in 1970. 
Although the projects reviewed comprise only about one-fourth of the total 
number of ongoing projects, the sample does give some idea of the state of 
evaluation in all the projects. iResults are generally disappointing and seem 
to bear out the lack ot expertise in evaluation found in the more general 
literature of Social Studies evaluation. Only one project, the Holt program 
at Carnegio-Mellon University, seems to have used a fairly comprehensive 
and systematic form of evaluation. In this case, the Stevenr-Morrissett 
System was used, and it is descriptive in nature and subject to the earlier- 
mentioned weaknesses. The Stevens-MorrJssett scheme is reportedly being 
used in other projects also. Comprehensiveness in curriculum evaluation 
design definitely seems to be lacking. Mauy of the projects do not seem 
to be able to produce any meaningful results from the evaluations that have 
been done. At least one project's procedures were of highly dubious 
validity, and one has tl:e feeling that several project staffs are groping 
in the dark when it comes to evaluative procedures. It should be noted in 
all fairness, however, that some of the data from the evaluations in projects 
has yet to be published. But, while some projects have not reached the 
stage where valid summative evaluation can be produced, the formative 
evaluation seems to be of poor quality and of little value. In addition, it 
seems that none of the projects have been compared with each other, and 
no standard criteria have emerged as yet which might aid comparative 
judgments. Assessments tend to be overly descriptive, the decision-making 
process is neglected, and informal and subjective evaluation procedures 
seem to be used far too regularly. 

It could be concluded that Social Studies curriculum developers still 
have much to learn about evaluation. In many cases the only criteria of 
whether a project is "good" or not is how many books have been sold by 
the project's publisher. For many projects evaluation has definitely not 
risen above the level of "bookkeeping. " 
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EVALUATION IN THE SOCIAL STUDIES: THE INTERNATIONAL PERSPECTIVE 

The amount of information to be four.d about evaluation of Social Studies 
curricula in countries other than the United States is extremely slim. The 
information which is available, however, seems to indicate that the develop- 
ment of Social Studias and the evaluation of Social Studies curricula in other 
nations lags further behind developments in other subject areas than it does 
in tnis country. The improvements which have occurred have come about 
through the auspices oi curriculum projects VvUthin a few nations and also 
through the efforts of international organizations. Here I shall briefly look 
at the work of these organizations and then at Social Studies and Social 
Studies evaluation in four countries: Britain, Australia, Canada, ar,d India. 

International O'^qanizations and Social Studies Evaluation 

An organization which works on the problems of curriculum development 
and evaluation on an international level is the International Association for 
the Evaluation of Educational Achievement (I.E. A.) based at the V/enner-Gren 
Center in Stockholm, Sweden. During the summer of 1971, the Association 
sponsored and administered a six-week International Seminar for Advanced 
Training in Curriculum Development and Innovation in Granna, Sweden. 
Approximately 20 countries sent teams of six persons each to the Seminar 
t3 study such problems as the determination of the specifications for curriculum, 
tne development of learning material and instructional procedures, the 
evaluation of the effectiveness of learning materials and instructional materials, 
and the in-service and pre-service training of teachers for curriculum changes. 
Benjamin Bloom directed the Seminar and also served as a member of the core 
faculty. Dr. Edwin Fenton served as the United States representative on 
Social Studies education. 

UNESCO, another international organization, is also struggling with the 
problems of educational evaluation. UNESCO is trying to develop a 
"Standardized Data Reporting System" for the evaluation of its functional 
literacy programs. These programs, some of which have been in operation 
for three or more years, have never been evaluated. In addition, UNESCO 
has sponsored seminars on curriculum evaluation in several countries, the 
most widely reported of which was the seminar on the teaching of the social 
sciences at the secondary level at Burwood Teachers College in Victoria, 
Australia. The impact of this effort will be examined shortly. 
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Social Studies Evaluat ion i n Other Nations 

Among the individual nations v;hich have made and continue to make 
some detennined efforts at improving their Social Studies curricula are 
Great Britain, Australia, Canada, and India. In Great Britain, the Nuffield 
Humanities Project has shown concern for the promotion of curricula centered 
on controversial issues^ but little evaluation information seems to be 
available. In the general area of curriculum evaluation, the British seem 
to be lagging behind the United States in their thinking on evaluation and 
curriculum development. V/hitfield and Kerr found a marked absence in 
Britain "of examples in wwich trial pupil achievements have been measured 
in terms of specific curriculum objectives, an operation we term 'hard 
evaluation. '"1 

Stephan Wiseman, in a review of British curriculum development and 
curriculum evaluation, expressed his anxiety over the apparent lack of 
enthusiasm of various educational organizations and committees for pursuing 
curriculum evaluation. He warned that: 

. . . evaluation is an essential part of the process of 
curriculum development: if it is ignored then the new 
syllabuses, produced with hope and optimism, will 
become entrenched as firmly as those they replace and 
the end result will be the substitution of a new set of 
claims for the imprisonment of the teachers and their 
pupils. The present wave of curriculum reform has 
occurred, somewhat belatedly, in response to demands 
of a rc.^idly changing environment. Such change will 
undoubtedly continue, and our new curricula must be 
capable of adpatation to continuing change. 2 

In Australia, too, concerted efforts are being made to improve Social 
Studies curricula. An important milestone in Social Studies improvement was 
the UNESCO seminar on the teaching of the social studies at the secondary 
level held at Burwood Teachers College in Victoria in 1S67. D. G. Duity, 
in a recent article, reviewed some of the curriculum changes which have 
occurred since the seminar. The Burwood seminar revealed widely diverging 

^R. C. Whitfield and J. F. Kerr. **Sorne Problems in Course Evaluation. " 
Teachers College Record . 72:267 (December 1970). 

2 

Stephan Wiseman. "Curriculum Development and Curriculum Evaluation. 
Researc h in Education fManrhest^^r University Press], 1:6 (May 1969), 
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views concerning the teaching of the social sciences, and it also ravcaled 
"the conservative nature of social science education i.n Australia compared 
with overseas countries. "3 The innovations v/hich have occurred since 
Burwood have varied among the Australian states. The influence of American 
curriculum developers, particularly Fenton and Taba. appear in the changes 
made in some of the states. Dufty's article concludes, however, that social 
science curriculum development in Australia remains "an adoptive, adaptive, 
or eclectic process, and no attempt has yet been made to develop an entirely 
new model for a social science course, v/ith Ihe possible exception of the 
social science curriculum development at Monash University. Overseas 
visitors would find little here that is unique or well-researched. "^ Other 
writers indicate that educational evaluation procedures are generally 
undeveloped as well. 

Canadian educators are showing increasing concern for evaluation as 
evidenced by the fact Lhat jn the fall of 1970, 251 educators enrolled in 
special two-day "evaluatjor research" clinics which travelled from Vancouver 
to Regina, Toronto, and Halifax under the auspices of the Canadian Edu- 
cational Researchers' Association. 5 Each two-day s-^ssion aimed to give a 
comprehensive picture of evaluation research through the presentation of 
papers by the clinic team as well as practice sessions which stressed 
methodology, statistical treatment of results, comparative methodologies, 
the analysis of current evaluation models, and the management of large 
projects via PERT. Garnet L. McDiarmid, writing on "The Meaning of 
Curriculum, " has stressed that "evaluation of the curriculum becomes an 
evaluation of the general effect of the various inputs and not just an 
evaluation of a specific course. McDiarmid also calls for clarification 
of terms, particularly the word "curriculum. " Better clarification of the 
term "evaluation" is also needed, he feels. Norman France, writing in 
Education Canada on "Evaluation in the High School: Fact or Fiction, "7 
equates evaluation with testing, a confusion of terminology discussed 
earlier. There seems to be a little in the way of evaluation procedures aimed 
specificai . at the Social Studies in Canada. 



3 

D. G. Dufty. "After Burwood What? A Study in Curriculum Innovation 
and Evaluation. " .Australian Jqurnai of_ Education. 15:75 (March 1971). 
Ibid . , p. 93. 

5 

Peggy Koopman. "What do We Mean by Evaluation ResearchJ' Education 
Canada. 11:48 (March 1971). 

Garnet McDiarmid. "The Meaning of Curriculum. " Ed ucation Canada. 

11:31 (March 1971). 

7 

Norman France. "Evaluation in the High School: Fact or Fiction. " 
Education, Canada, 11:14-18 (March 1971). 
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The concept of Social Studies is still quite vague in India. P. K. 
Khasnavis has noted that Indian scholars talk about Social Studies, but it 
is doubtful if a common definition of Social Studies is shared by everyone. 
Indian educators are, nevertheless, now showing increasing interest in the 
field. 8 There is, for example, an informative little journal called Social 
Studies Teacher published by the University of Baroda. Moreover, a number 
of articles proposing reforms in Indian thinking about Social Studies have 
appeared in the journal in recent years. In addition, it would seem that 
Indian educators are beginning to look at American work in Social Studies. 
Since the importance of Social Studies as a subject is only beginning to be 
recognized, evaluation of Social Studies curricula is quite undeveloped. It 
is hoped that as the Indian educators develop Social Studies curricula, they 
will include evaluation as an integral part of the curricula from the outset. 

It can be seen then that both Social Studies and o^-^ial Studies evaluation 
in other nations are considerable behind the work currently being done in the 
United States. The efforts of the international organizations such as UNESCO 
would seem to be one of the most important means of rapidly improving Social 
Studies in other countries. But it v/ould also seem heloful if curriculum 
developers outside of the United States became better acquainted with the 
work of Provus, Guba, Stufflebeam, Stake, and otner major American figures 
in the field of curriculum evaluation with the hope that evaluation in any 
subject area, including the Social Studies, could be rapidly improved 



8 

P. K. Khasnavis. **Meaning of Social Studies According to American 

Scholars." Social Studies Teacher [University of Baroda, India], 7-5 
(July 1970). ... J . 



CONCLUSION 



Evaluation in the Social Studies is still, for the most part, at the level 
of bookkeeping. Most of the work on curriculum evaluation in the Social 
Studies has taken place in the United States, usually in curriculum projects 
financed from varying sources. Both the general literature on evaluation in 
thp Social Studies and the evaluation efforts in the curriculum projects reveal 
the failures and weaknesses of cuniculum evaluation in this subject area. 
Evaluation of Social Studies curricula in other nations remains considerably 
behind the work being done in the Unitad States. 

The Social Studies are changing and moving at long i : after many 
years of stagnation. But if the field is to expand and dev^.^p in a systematic 
and intelligent manner, then a parallel and dynamic effort must also come 
about in Social Studies evaluation. That evaluation must not be something 
haphazardly "added on" to Social Studies development programs but must be 
an integral part of chose programs from their inception. Perhaps, then. 
Social Studies will stop being the laggard among the disciplines and become 
more cohesive and worthwhile. "Social Studies, " "Curriculum, " and 
"Evaluation" are all difficult to define at the moment but evaluation may be 
the key to improvement of the curricula in the Social Studies as well as in 
other fields. 
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L AddiUonal Social Studies Projects: 

Asian Studies Curriculum Project 

University of California at Berkeley 

Committee on the Study of History 

Amherst College, Amherst, Massachusetts 

Economics Curriculer iv4aterials for Secondary Schools 
Ohio University 

Elementary Economics Project 

University of Chicago, Industrial ^.elations Center 

Elementary Social Science Ed ucation Program 

University of Michigan, Ann Arbor, Michigan 

Harvard University Social Studies Project 

Harvard University, Cambridge, Massachusetts 

Materials and Activities for Teachers and Children (MATCH) 
Boston Children's Museum, Boston, Massachusetts 

Our WorkinC) v/orld 

University of Colorado 

Social Studies Curriculum Project 

Carnegie-Mellon University, Pittsburgh, Pennsylvania 

Sociological Resources for Secondary Schools 
American Sociological Association 

Taba Social Studies Curriculum Project 

San Francisco State College, San Francisco, California 

Project Directors: 

Angell, Robert C. 

Sociological Resources for the Social Studies 

Brown, Richard H. 
Amherst College 

Collier, Malcolm 

Anthropology Curriculum Study Project 



Darcy, Robert L. 

Ohio UnivGrsity 



Dow, Peter 

Education Development Center, Inc. 

English, Raymond 

Educational Research Council of America 

Fenton, Edvv^in 

Carnegie-Mellon University 

Fox, Robert S. 

University of Michigan 

Halsey, Van R. 

Amherst College 

Kresse, Frederick H. 

Boston Children's Museum 

Lippitt, tonald 

University of Michigan 

Lovenstein, Meno 
Ohio University 

Michaells, John U. 

University of California at Berkeley 

Newmann, Fred M. 

Harvard University 

Oliver, Donald V/. 

Harvard University 

Parsons, T. William 

Anthropology Curriculum Study Project 

Powell, Phillip E. 
Ohio University 

Rader, V/iUiam D. 

Univeisity of Chlrago, rn.-^iistrial Relations Center 
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Ratcliffe, Robert 

Lav\r in American Society Foundation 

Senesh, Lawrence 

University oi Colorado 

Wallen, Norman E. 

San Francisco State College 

III. Project Materials: 

Four Communities Around the World (Grade 3) 
San Francisco State College 

From Subject to Citizen 

Education Development Center, Inc* 

Geography in an Urban Age 

High School Gecgro.phy Project 

History as Culture Change: An Overview 
Anthropology Curriculum Study Project 

House of Ancient Greece 

Boston Children's Museum 

Humanities in Three Cities: An Inquiry Approach 
Carnegie-Mellon University 

Introduction to the Behavioral Sciences: An Inquiry Approach 
Carnegie-Mellon University 

The Japanese Family 

Boston Children's Museum 

Justice in Urban America Series 

Um in American Society Foundation 

Man: A Course of Study 

Education Development Center, Inc. 

Manpower and Economic Education: Opportunities in American 
Economic Life 

Ohio University 
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Middle and SouLn America: Sociedes in Transition (Grade 6) 
San Francisco State College 

Neighbors ac V^ork 

University of Colorado 

N ew Hisrory of the United States: An Inquiry Approach 
Carnegie-Mellon University 

Our State: ^ Changing Society (Grade 4) 
San Francisco State College 

Readings in Sociology Series 

Sociological Resources for the Social Studies 

Shaping ol Western Society: An incruiry Approach 
Carnegie-Mellon University 

Social Science Laboratory Units 
University of Michigan 

Tradition and Change in Four Societies: An Inquiry Approach 
Carnegie-Mellon University 

United States and Canada: Societies in Transition (Grade 5) 
San Francisco State College 

U_aii:ed Scates; Change, Problems, and Promises (Grade 8) 
San Francisco State College 

Units in American History 

Amherst College, The Committee on the Study of History 

Western Ci^/ilisation; Perspectives on Change (Grade 7) 
San Francisco State College 

World Studie s I n quiry Serie s 

University of California at Berkeley 
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Alkin, Marvin C. "Evaluation of Theory Development. " Evaluation Comment 
(October 1969), p. l. " 

Evaluation - is the process of: 

1. Ascertaining the decision areas of concern 

2. Selecting appropriate information 

3. Collecting and analyzing information in order 
to report summary data useful to decision- 
makers in selecting among alternatives. 

Evaluation Need Areas: 

1. Systems Assessment 

2. Program Planning 

3. Program Implementation 

4. Program Improvement 

5. Program Certification 



Astin, Alexander W. and Panos, Robert J. "The Evaluation of Educational 
Programs. " in Educational Measurement (ed. by Robert L. Tnorndike). 
Washington, D. C. : 1971, pp. 733. 

Evaluation involves the collection of inlormation concerning the 
impact of an educational program. An educational program is 
conceiV3d as any ongoing educational activity which is designed 
to produce specified changes in the behavior of the individuals 
who are exposed to it. The major function of evaluation is to 
provide the decision maker with relevant information about the 
inputs, outputs, and operations of the program under construction. 



Brooks, M- "The Community Action Program as a Setting for Applied Research. 
Journal of Applied Beh avioral Science . No. 1 (1965), p. 337. 

The procedure by vv^hich programs are studies to ascertain their 
effectiveness in the fulfillment of goals- 



^=^ro, Francis G. "Issues in the Evaluation of Social Programs. " Review of 
Educ ational 'Research . Vol. 41, No. 2 (April 1971). 

Page 87: 

... the procedure by which programs are studied to ascertain 
their effectiveness in the fulfillment of goals. " (from B. G. 
Greenberg. "Evaluation of Social Programs. " Review of the 
Internation al Statistical Insti tute p. 36). 



Page 88: 

. . . methodological activity which combines performance 
data with a goal scale. (Michael Scriven. "The Methodology 
of Evaluation. " AERA Monograph Series on Curriculum Evaluation 
No. 1, Perspectives of Curriculum Evaluation, 1967) 

Cohen, David K. "Politics and Research: Evaluation of Social Action 
Programs in Education. " Review of Educational Research , Vol. 40, 
No. 2 (1970), pp. 214, 232. ^ 

Evaluation is a mechanism with v/hich ihe character of an 
educational enterprise can be explored and expressed. . . . 
Evaluation is a technique for measuring the satisfaction 
of public priorities; to evaluate a social action program 
is to establish an information system in which the main 
questions involve the allocation of power, status, and 
other public goods. 



Cronbach, Lee J. "Course Improvement Through Evaluation. " Teachers 
College Record, Vol. 64, No. 8 (May 1953), p. 672. 

Evaluation may be defined as the collection and use of 
information to make decisions about an educational program. 

Denny, Terry (ed. ). "Educational Evaluation. " Review of Educational 
Research, Vol. 40, No. 2 (April 1970), p. 181. 

Evaluation— is the gathering of empirical evidence for decision- 
making and the justification of decision-making policies and 

the values upon which they are based The goal of evaluation 

must be to answer questions of selection, adoption, support 
and worth of educational materials and activities. ... In the 
past we have avoided the goal of evaluation with its inherent 
threat to teachers, administrators, and curriculum developers 
and have concentrated on one or more of the non-threatening 
roles evaluation can play. 



iiuy, Terry, Educational P>-o_duct r< eport. Vol. 2, No. 5 (February 1959). 

Evaluation is a worrisome word in educational parlance 
that resists definition about as stoutly as any concept 
in vogue. 



<8 

Doll, Ronald C. Curriculum Improvements Boston: AUyn and Bacon 
1970, p. 379- 

Evaluation may be defined as a broad and continuous effort 
to inquire into the affects of utilizing educational content 
and process according to clearly defined goals. 



Glass, G* "The Growth of Evaluation Methodology* " AERA Monograph 
Series on Curriculum Evaluation, No. 7, p. 2. 

An attempt to assess the worth or social utility of a thing. 



Gronlund, Norman E. Measurement and Evaluation of Teaching , 
New York: MacMillan Company, 1970, 2nd, ed. 

Evaluation may be defined as a "systematic process of 
determining the extent to which educational objectives 
are achieved hy pupils. " 



Cuba, Egon G, _Evaluation and Change in Education , A paper prepared 
for the Elk Grove Training and Development Center Spring Evciluation 
Conference, Arlington Heights, Illinois, May 16, 1968, p, il. 

Evaluation. . , is a process of providing and using information 
for making educational decisions, . , this definition sees 
evaluation as continuing (cyclical and looping), multi- 
faceted (involving many different methods and techniques), 
practical, and relevant. 



Karris, Chester, "Some Issues in Evaluation. " The Speech Teache r 
12: 191 (1953), ' 

Evaluation is. , , the systematic attempt to gather evidence 
regarding student behavior that accompanies planned 
educational experiences. 



Hayes, S. "Evaluating Development Projects. " Paris: UNESCO, 1959, 

A body of concepts and practices which have proved their 
usefulness (in the field of social sciences) and which are 
applied in such a way that they can contribute to the 
improvement of practical activities. 
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Kresh, Esther. '*An Overview of the Discrepancy Evaluation Model and 

a Related Case Study. Presented at the Ohio Conference on Evaluation, 
July 28-30, 1S59. 25 p. (mimeo. ) 

Evaluation is the process of providing decision makers with 
relevant and timely information for making decisions. 



Merriman, Howard O. Educational Evaluation and Decision-MaV ing. 

New York: Phi Delta Kappa National Study Committee on Evoluation 
at the /^ERA Symposia on Educational Evaluation, February 1971, p. 1. 

Definition: EDUCATIONAL Evaluation is the Process of 
Delineating, Obtaining and Providing Useful 
Information for Judging Decision Alternatives. 

Key Terms: 



Process - A particular and continuing activity subsuming many 

methods and involving a number of steps or operations. 
Decision Alternatives - Two or more different actions that 

might be taken in response to some 
situation requiring altered action. 
Information - Descriptive or interpretive data about entities 
(tangible or intangible) and their relationships, 
in terms of some purpose. 
Delineating - Identifying evaluative information required through 
an inventory of the decision alternatives to be 
weighed and the criteria to be applied in v^/eighing 
them. 

Ob taining - Making information available through such processes 
as collecting, organizing, and analyzing and through 
such Tormal means as measurement, data processing, 
and statistical analysis. 
Providing - Fitting information together into systems or sub- 
systems that best serve the purposes of the 
evaluation and reporting the information to the 
decision-maker* 
Useful - Satisfying the scientific, practical, and prudential 
criteria (as specified in Chapter 1 of the Monograph) 
and pertaining to the judgmental criteria to be 
employed in choosing among the decision alternatives. 
Juc^ging - The act of choosing among several decision 
n^ternatlves, the art of dArision-making. 
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Scrlven, MichaeL "The Methodology of Evaluation/' in Perspectives of 
Curriculum Evaluation . AERA Monograph Series No. 1, 1967, p. 43. 

Formative Evaluation. . . is evaluation to improve a program 
during its course 

Summative Evaluation . . . is evaluation designed to appraise 
a program after it is established. 



s, David L. (ed. ) "Evaluation .^e^earch. ' International Encyclopedia 
of the Social Sciences^ Vol. 5. New York: The MacMillan Company and 
Free Press, 1968, p. 198. 

The primary purpose of evaluation research is to "provide 
objective, systematic and comprehensive evidence on the 
degree to which the program achieves its intended objectives 
plus the degree to which it producer- other unanticipated 
consequences". . . 



Stufflebeam, Daniel L. "Toward a Science of Educational Evaluation. " 
Educational Technology . Vol. 8 (July 30, 1968), p. 5. 

Evaluation means the provision of information through f ormal 
means, such as criteria, measurement and statistics, to 
serve as rational bases fo r making judgements in decision 
situations. A decision is a choice among alternatives. A 
decision situation is a set of alternatives. Judgment is 
the assignment of values to alternatives. A cri^.^rion is a 
rule by which values are assigned to alternatives, and 
optimally such a rule includes the specification of 
variables for measurement and standards for use in judging 
that which is measured. Statistics is the science of 
analyzing and interpreting sett, of measurements. And, 
Tieasurement is the assignment of numerals to entities 
according to rules, and such rules usually include the 
specification of sample elements, measuring devices and 
conditions for administering "nd scoring the measuring 
devices. Stated simply, eval uation is the science of 
providing information for dec i sion-makino . 

Stufflebeam, Daniel L. "The Use and Abuse of Evaluation in Title III. " p. 129. 

More specifically, evaluation is defined herein as the process 

of acquiring and using information for making decisions associated 

with planning, programming, and recycling Program activities. 
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Suchman^ Edv/aro. Evaluaiive Rssearch . Nev^ York: Russell Sage Foundati 
Page 31: 

. . . tha determination (whether based on opinions, records, 
subjective, or objective data) of the results (whether 
desireable or undesireable; transient or permanent; immediate 
or delayed) attained by some activity(wnether a program or 
part of a program, a drug or a therapy, an ongoing o.- a one- 
shot approach) designed to accomplish some valued goal or 
objective (wnecher ultimate, intermediate, or immediate, 
effort or performance, long or short range). 

This definition contains four ksy dimensions: process — the 
"determination, " crireria — the"resuUs. " stimulus — the activity, 
and value- -the objective. 



Page 31: 

. . . we would like to propose a distinction between evaluation . . . 
as the general process of judging the wort.iwhileness of some 
activity regardless of the method employed, and evaluative 
Lesearcli as the specific use of the scientific method for the 
purpose of making an evaluation. 



Page 28: 

. . .the process of determining the value or amount of success 
in achieving a predetermined objective. It includes at least 
the following steps: formation of the objective, identification 
of the proper criteria to be used in measuring success, 
determination and explanation of the; degree of success, 
recommendations for further program activity, (from Glossary 
olMnU'listrative. Terms i n Public Health. Vol. 50 (February 
1960), pp. 225-226) 



Page 28: 

. . . the measurement of desirable and undesirable consequences 
of an action that has been taken in order to forward some goal 
v/e value, (from Riecken, Henry v;. The Volunte er ^Vork Camp : 
A_Psychgiqgicalj;vaJuation. Cambridge, Mass. : Addison- 
V'/esley Press, 1952, p. 4) 
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Page 29: 

. . - the procedures of fact-finding about the results of planned 
social action. (Hymaa, Herbert K. , et al. Applications of 
J/ethods of Education: Four Studies of the Encampmen ^or 
Citizenship , Berkeley: University of California Press, 1S52, 
p. 3) 



Page 2G: 

Program evaluation can be defined as the measurement of 
success in reaching a stated objective, (from James, George. 
"Planning and Evaluation in Health Programs. " in Administration 
of Community Health Services . Chicago: International City 
Managers Associacion, 1961, p. 12<1) 

Page 30: 

... a process v/hich enables t'le administrator to describe 
the effects of his programme, and thereby to make progressive 
adjustments in order to reach his goals more effectively, {trom 
Klineberg, Otto. The Problem of Evaluacio' . " International 
Social Science Bulletin. Vol. 7, No. 3 (1955), pp. 346-352) 

Page 11: 

An evaluation is basically a judgment oi worth — an appraisal 
of value 

Taba, Hilda. C urriculum Development. New York: Harcourt, Brace, and 
World, 1962, p. 312. 

Evaluation is the process of determining what these changes 
(in education) are, and appraising them against the values 
repiesented in objectives to find out how far the objectives 
in education are being achieved. 



Tumin, Melvin M. "Evaluation of the Effectiveness of EducaUon: Some 
Problems and Prospects. " Interchange, Vol. 1, No. 3 (1970), p. 96. 

Evaluation means different things to different people, not only 
because they are defensive about the possible results of a 
systematic scrutiny of their effectiveness, but also because 
they have different notions as to what are legitimate sources 
of pride and shams. . . . Evaluation is a many-edged s vord. 

O 
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It must unavcidabl/ threaten a ad damage some of the actors 
in the scene, just as unavoidably it will improve the power 
position of otheis. The negative as v/ell as the positive 
function^ of evaluation must be seen and understood hy the 

evaluator the process is one of discover/ for the actors 

subject to evaluation and for the professionals engaged in 
the act. The latter must always be prepared to revise the 
initial models, for technical reasons, first, and because, 
second, they must, if they are at all sensitive, come to 
learn things about the connections among inputs, processes, 
and outcomes that tney did not know before, no matter how 
experienced they may be. 



Westbury, Ian. "Curriculum Evaluation. " Review of Educationa l Research , 
Vol. 40, No. 2, 0. 240. " 

Curriculum evaluation is. . . a body of techniques, methodologies 
and principles created deliberately (and recently) to give some 
systematic form to the ways in which the assertion "we must 
evaluate" can be made to work. 
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